[aoti-backend-consolidation 2/3] backend.py #15528

Gasoonjia · 2025-11-03T17:51:08Z

Summary:

Summary

This diff consolidates the backend functionality into a single target //executorch/backends/aoti:aoti_backend and simplifies the cuda backend target by making it dependent on the consolidated backend target.

The following changes are made in this diff:

Creation of a new target //executorch/backends/aoti:aoti_backend in fbcode/executorch/backends/aoti/targets.bzl which includes the necessary dependencies for the AOTI backend.
Update of the //executorch/backends/cuda:cuda_backend target in fbcode/executorch/backends/cuda/TARGETS to depend on the new //executorch/backends/aoti:aoti_backend target instead of individual AOTI backend dependencies.
Creation of a new file fbcode/executorch/backends/aoti/aoti_backend.py which imports the necessary dependencies and passes for the AOTI backend.
Simplification of the xplat/executorch/backends/cuda/cuda_backend.py file by removing unnecessary imports and using the new AotiBackend class from the aoti_backend.py file.
ghstack-source-id: 319556735

Reviewed By: larryliu0820

Differential Revision: D85704977

pytorch-bot · 2025-11-03T17:51:12Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15528

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 1 Cancelled Job

As of commit 57e8ffe with merge base d2c011e ():

NEW FAILURES - The following jobs have failed:

Apple / build-demo-ios / macos-job (gh)
pull / unittest / linux / linux-job (gh)
exir/backend/test/test_lowered_backend_module.py::TestBackendAPI::test_emit_nested_lowered_backend_module
pull / unittest-editable / linux / linux-job (gh)
exir/backend/test/test_lowered_backend_module.py::TestBackendAPI::test_emit_nested_lowered_backend_module
Test CUDA Builds / export-model-cuda-artifact (openai, whisper-small, quantized-int4-tile-packed) / linux-job (gh)
RuntimeError: Command docker exec -t 7fdc1ca6b4cb3c190bdd35008d2ee69f15f21f4dc085e533b26ef773b0267c6e /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

pull / unittest-editable / macos / macos-job (gh)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

meta-codesync · 2025-11-03T17:51:26Z

@Gasoonjia has exported this pull request. If you are a Meta employee, you can view the originating Diff in D85704977.

larryliu0820

Review automatically exported from Phabricator review in Meta.

github-actions · 2025-11-03T17:52:03Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

manuelcandales · 2025-11-05T20:50:12Z

backends/apple/metal/metal_backend.py

-            blob_data = f.read()
-
-        named_data_store.add_named_data(
-            method_name + "_weights_blob", blob_data, 1, "aoti_metal_blob"


what's the new name?

here's the new name: https://github.com/pytorch/executorch/pull/15528/files#r2496118034

ahh i called it mps now instead of metal; thanks let me solve it

Gasoonjia · 2025-11-05T20:52:40Z

backends/aoti/aoti_backend.py

+        named_data_store.add_named_data(method_name + "_so_blob", so_data, 1, None)
+        weights_blob_data_type = f"aoti_{device_name}_blob"
+        named_data_store.add_named_data(
+            method_name + "_weights_blob", blob_data, 1, weights_blob_data_type


here's the blob name in the new setting @manuelcandales

manuelcandales · 2025-11-05T20:56:11Z

backends/aoti/aoti_backend.py

+
+        # Add SO and weights blob separately
+        named_data_store.add_named_data(method_name + "_so_blob", so_data, 1, None)
+        weights_blob_data_type = f"aoti_{device_name}_blob"


yeah, this will now be aoti_mps_blob, so, need to update this in metal.yml (uses aoti_metal_blob)

cccclai · 2025-11-05T22:04:04Z

exir/backend/backend_api.py

+                subclasses.update(_get_all_final_backend_details_subclasses(subclass))
+        return subclasses
+
    backend_name_to_subclass = {


I remember it was discussed at the beginning of the delegate meeting - the backend implementation should be final and we didn't want to get involve in nested backends

executorch/exir/backend/backend_api.py

Line 111 in 3e9629a

# All backend implementation are final, so we don't need to consider nested subclasses.

it's hard to guard in python though

In the example we put, we did try to put a final

executorch/exir/backend/test/backend_with_compiler_demo.py

Line 28 in 3e9629a

@final

If there are shared logic, maybe put up a shared folder?

Thanks for comments. I make the AOTIBackend as a collector of sharing logics but not a actual backend, and make CUDABackend and MetalBackend inherit from both AOTIBackend and BackendDetails to avoid making any update on backend_api.py.

cccclai · 2025-11-14T19:16:46Z

backends/aoti/aoti_backend.py

+@experimental(
+    "This API and all of aoti-driven backend related functionality are experimental."
+)
+class AotiBackend(ABC):


Is AotiBackend an actual class or an abstract class?

After reading, it seems like we will just have metal backend and cuda backend as the actual backend?

yes it is just a abstract class not a real backend for meet backend_details' requirement

cccclai · 2025-11-14T19:18:36Z

setup.py

-
            if is_linux_x86():
                os.environ["EXECUTORCH_BUILDING_WHEEL"] = "1"
+                from backends.qualcomm.scripts.download_qnn_sdk import _download_qnn_sdk


Mengwei has this PR #15546 maybe can coordinate how to land together or maybe after he merged his change

have reverted my changes

Summary: pytorch#15528 initially wanted to subclass a backend.. It was currently already guarded by https://github.com/pytorch/executorch/blob/main/exir/backend/backend_api.py#L111-L112 meaning that subclass will not show up. However it's not super obvious so we want to guard by disallowing subclass at all Differential Revision: D87105211

Summary: pytorch#15528 initially wanted to subclass a backend.. It was currently already guarded by https://github.com/pytorch/executorch/blob/main/exir/backend/backend_api.py#L111-L112 meaning that subclass will not show up. However it's not super obvious so we want to guard by disallowing subclass at all Reviewed By: Gasoonjia Differential Revision: D87105211

Summary: #15528 initially wanted to subclass a backend.. It was currently already guarded by https://github.com/pytorch/executorch/blob/main/exir/backend/backend_api.py#L111-L112 meaning that subclass will not show up. However it's not super obvious so we want to guard by disallowing subclass at all Differential Revision: D87105211

Copilot

Pull Request Overview

This PR consolidates AOTI (AOT Inductor) backend functionality by creating a shared base class (AotiBackend) that can be reused across different device backends (CUDA, Metal). The refactoring reduces code duplication and standardizes the compilation workflow.

Key Changes:

Created a new AotiBackend mixin class that provides common AOTI compilation logic
Refactored CUDA and Metal backends to inherit from AotiBackend instead of duplicating code
Updated build targets to reference the consolidated backend dependency

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
backends/aoti/aoti_backend.py	New base class providing common AOTI compilation functionality
backends/cuda/cuda_backend.py	Simplified to use `AotiBackend` with CUDA-specific configuration
backends/apple/metal/metal_backend.py	Simplified to use `AotiBackend` with Metal-specific configuration
backends/aoti/targets.bzl	Added build target for the new `aoti_backend` module
backends/cuda/TARGETS	Updated to depend on consolidated `aoti_backend` target
setup.py	Moved platform detection function to avoid import errors
examples/models/whisper/CMakeLists.txt	Updated target name from `aoti_cuda` to `aoti_cuda_backend`
examples/models/gemma3/CMakeLists.txt	Updated target name from `aoti_cuda` to `aoti_cuda_backend`
extension/llm/tokenizers	Updated subproject commit reference

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

backends/aoti/aoti_backend.py

Copilot · 2025-11-17T16:16:12Z

backends/apple/metal/metal_backend.py

+    @staticmethod
+    def get_supported_fallback_kernels() -> Dict[str, Any]:
+        return {
+            "aoti_torch_mps_addmm_out": None,


The kernel 'aoti_torch_mps_addmm_out' was added to the supported fallback kernels but was not present in the original code. Verify this kernel is intentionally supported and not an accidental addition during refactoring.

Suggested change

"aoti_torch_mps_addmm_out": None,

Summary: This diff consolidates the backend functionality into a single target `//executorch/backends/aoti:aoti_backend` and simplifies the cuda backend target by making it dependent on the consolidated backend target. The following changes are made in this diff: * Creation of a new target `//executorch/backends/aoti:aoti_backend` in `fbcode/executorch/backends/aoti/targets.bzl` which includes the necessary dependencies for the AOTI backend. * Update of the `//executorch/backends/cuda:cuda_backend` target in `fbcode/executorch/backends/cuda/TARGETS` to depend on the new `//executorch/backends/aoti:aoti_backend` target instead of individual AOTI backend dependencies. * Creation of a new file `fbcode/executorch/backends/aoti/aoti_backend.py` which imports the necessary dependencies and passes for the AOTI backend. * Simplification of the `xplat/executorch/backends/cuda/cuda_backend.py` file by removing unnecessary imports and using the new `AotiBackend` class from the `aoti_backend.py` file. ghstack-source-id: 319556735 Reviewed By: larryliu0820 Differential Revision: D85704977

meta-codesync · 2025-11-20T23:20:34Z

@Gasoonjia has imported this pull request. If you are a Meta employee, you can view this in D85704977.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Gasoonjia requested review from JacobSzwejbka, cccclai, larryliu0820 and shoumikhin as code owners November 3, 2025 17:51

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 3, 2025

meta-codesync bot added fb-exported meta-exported labels Nov 3, 2025

larryliu0820 approved these changes Nov 3, 2025

View reviewed changes

Gasoonjia changed the title ~~backend.py (#15430)~~ [aoti-backend-consolidation 2/3] backend.py Nov 3, 2025

mergennachin requested a review from manuelcandales November 3, 2025 18:06

Gasoonjia requested review from jackzhxng and mergennachin as code owners November 4, 2025 06:23

manuelcandales reviewed Nov 5, 2025

View reviewed changes

Gasoonjia commented Nov 5, 2025

View reviewed changes

manuelcandales reviewed Nov 5, 2025

View reviewed changes

cccclai reviewed Nov 5, 2025

View reviewed changes

Gasoonjia requested a review from kirklandsign as a code owner November 10, 2025 23:51

manuelcandales approved these changes Nov 11, 2025

View reviewed changes

cccclai reviewed Nov 14, 2025

View reviewed changes

cccclai mentioned this pull request Nov 14, 2025

Prohibit nested backends #15831

Merged

mergennachin requested a review from Copilot November 17, 2025 16:11

Copilot AI reviewed Nov 17, 2025

View reviewed changes

Gasoonjia and others added 12 commits November 20, 2025 14:18

solve qualcomm import issue

e98d339

Update metal.yml

05e081f

Update metal.yml

37ba819

Update metal.yml

7bd8781

recover metal workflow

4761731

Update metal device name

dc1fd12

Update metal device name in metal_backend.py

b11a3fb

make aoti_backend not a real backend

3b05c52

swap inherit order

b83071f

run lintruner

c9d871a

solve ci

4be608d

merge lastest update

1e90fd0

Gasoonjia force-pushed the export-D85704977 branch from d129e1e to 1e90fd0 Compare November 20, 2025 23:10

Gasoonjia and others added 2 commits November 20, 2025 17:05

revert extra changes

48e8c86

Update backends/aoti/aoti_backend.py

57e8ffe

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Gasoonjia merged commit 9e7e17c into main Nov 21, 2025
161 of 168 checks passed

Gasoonjia deleted the export-D85704977 branch November 21, 2025 06:40

[aoti-backend-consolidation 2/3] backend.py #15528

[aoti-backend-consolidation 2/3] backend.py #15528

Uh oh!

Conversation

Gasoonjia commented Nov 3, 2025

Summary

Uh oh!

pytorch-bot bot commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15528

❌ 4 New Failures, 1 Cancelled Job

Uh oh!

meta-codesync bot commented Nov 3, 2025

Uh oh!

larryliu0820 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 3, 2025

This PR needs a release notes: label

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Copilot AI Nov 17, 2025

Choose a reason for hiding this comment

Uh oh!

meta-codesync bot commented Nov 20, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

pytorch-bot bot commented Nov 3, 2025 •

edited

Loading

This PR needs a `release notes:` label